Solution: Information Gain

Information Gain

Information Gain Formula

Note that the child groups are weighted equally in this case since they're both the same size, for all splits. In general, the average entropy for the child groups will need to be a weighted average, based on the number of cases in each child group. That is, for m items in the first child group and n items in the second child group, the information gain is:

\mathrm{Information\;Gain} = \mathrm{Entropy}(Parent) - \left[\frac{m}{m+n}\mathrm{Entropy}(Child_1) + \frac{n}{m+n}\mathrm{Entropy}(Child_2)\right]

Next Concept